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Abstract This paper investigates the necessary conditions of optimality for uni- 
formly overtaking optimal control on infinite horizon with free right endpoint. 
Clarke's form of the Pontryagin Maximum Principle is proved without the as- 
sumption on boundedness of total variation of adjoint variable. The transversality 
condition for adjoint variable is shown to become necessary if the adjoint variable 
is partially Lyapunov stable. The modifications of this condition are proposed for 
the case of unbounded adjoint variable. The Cauchy-type formula for the adjoint 
variable proposed by S.M. Aseev and A.V. Kryazhimskii in pQ,[2] is shown to 
complement relations of the Pontryagin Maximum Principle up to the complete 
set of necessary conditions of optimality if the improper integral in the formula 
converges conditionally and continuously depends on the original position. The 
results are extended to an unbounded objective functional (described by a non- 
convergent improper integral), unbounded constraint on the control, and uniformly 
sporadically catching up optimal control. 
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Introduction 

The Pontryagin Maximum Principle for infinite horizon problems has already been 
formulated in the monograph [26], but without the transversality condition the 
obtained relations were incomplete and in general, selected a much too broad 
family of potentially extremal trajectories. A significant number [20,4,8,25,31, 
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10,22 of such conditions has been proposed; however, as it was noted in, for 
example, 20,25,30 ,[4, Sect. 6].|28l Example 10.2], these conditions may be either 
inconsistent with the relations of the Pontryagin Maximum Principle, or follow 
from them. Hence the need to investigate the applicability of a transversality 
condition (see [4,8,22,25,31,36,28,29,27 ) and the need to separately check if it 
is necessary for a specific optimization problem. The first aim of this paper is 
to offer a common approach to selecting a necessary transversality condition on 
the adjoint variable for this problem (Subsect. I4.3tf01) . However, the necessity of 
a condition does not imply its nontriviality on solutions of the relations of the 
Maximum Principle. Hence the need to find a condition that would select a single 
solution of the relations of the Maximum Principle for any uniformly overtaking 
optimal control. In the papers 1,2,3,4,5 , Aseev and Kryazhimskii develop and 
investigate the Cauchy-type formula for the adjoint variable that possesses such 
a property. The second aim is to maximize the applicability of the approach [5] 
(Sect. [5]). 

First of all, we construct the bicompact extension (see [31]) for the space of 
admissible controls in the form of the inverse limit of the sequence of correspond- 
ing finite horizon extensions. It is shown that there exists a uniformly overtaking 
optimal generalized control for the case of a conditionally convergent objective 
functional that converges uniformly with respect to all trajectories; this general- 
izes some results [9] , [12] , [16] . Without this assumption, for uniformly overtaking 
optimal control for problems with free right endpoint, the necessity of the Pon- 
tryagin Maximum Principle in Clarke's form for the more general conditions than 
in [i) 5, 25 , [36, Theorem 2.1] is shown; the obtained result is not a part of the 
results 0[II27]. 

In Subsect. 13.31 for a free right endpoint problem, the convergence of transver- 
sality conditions on the adjoint variable is provided by the integral partial stability 
of the adjoint variable as a component of the Maximum Principle system. Thus 
we obtain the assumptions that guarantee the necessity of such condition, which 
are more general than the assumptions in [28]. [361 Corollary 2.1],J7J Theorem 1] 
(It seems that the first implementation of the approach that employs the notion 
of stability to obtain transversality conditions was in [31]). If the the objective 
function and the right-hand side of the equation of dynamics are smooth (in the 
phase variable) , then, instead of integral partial stability we can check the simpler 
condition of partial Lyapunov stability for the variable ip as a component of solu- 
tions of the system of the Maximum Principle. For example, we can check if all 
Lyapunov exponents are negative for this variable (see Subsect. I4.3[> . 

We propose the new transversality condition: the product of the adjoint vari- 
able and a matrix function of time must be vanishingly small at infinity. This 
condition becomes necessary if the product is stable. The stability can be pro- 
vided by the correct choice of the matrix function; the choice may also reflect 
a priori information on stability and asymptotic estimates, which may allow to 
pinpoint the single extremal (see Subsect. 14.41) . 

If the above matrix function is the fundamental matrix of linearized system 
along the optimal trajectory, then, the corresponding transversality condition auto- 
matically yields the formula that was proposed for affine systems in pQ,[2], general 
case of which was examined in [4, Theorems 11.1, 12.1], [5] Theorem 2], [71 Theo- 
rem 1]. As it was shown in [4, Sect. 16], the results of [8ll36] are the corollaries of 
[1 Theorem 12.1],[5l Theorem 2]. 
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Such choice of matrix function allows us to reduce the question of necessity of 
the corresponding condition to not just the question of the stability of the prod- 
uct, but even to the issue of checking if the improper integral from the formula [H 
(12.8)], [5, Theorem 2] , Theorem 1] converges conditionally and continuously de- 
pends on the initial position of the original problem. This yields the Cauchy-type 
formula for the adjoint variable and the "normal" Pontryagin Maximum Principle 
under the assumptions weaker than in [3| Theorem 12.1]. This result also general- 
izes |27l Theorem 3.2]. [361 Theorem 2.1] and [281 Theorems 3.1 and 8.1] (as far as 
the necessary conditions for problems with free right endpoint are concerned). 

For the case of monotonous system, we also demonstrated certain estimates 
for the adjoint variable. In particular, we obtained the nonnegativity of adjoint 
variable under the weaker assumptions than in [6, Theorem 1], [351 Theorem 1], 
H Theorem 10.1]. 

In the last part of the paper, we extend the obtained results to the the cases of 
<7-compact constraints on controls and to uniformly sporadically catching up opti- 
mal controls. Important breakthroughs for these problems were recently achieved 
in 0. 

A part of the results of this paper has been shown and announced in paper [23] . 



1 Preliminaries 

We consider the time interval T = R>o- The phase space of the original con- 
trol system is the certain finite-dimensional metric space X = R m . The unit ball 
of this space is denoted by D. Let L denote the linear space of all m x m ma- 
trices. For the sake of definiteness, let us equip L with the operator norm. The 
symbol E (which may be equipped with some indices) denotes various auxiliary 
finite-dimensional Euclidean spaces, and the symbol 13(E) denotes the a-algebras 
of their Borel subsets. 

For a subset A of a topological space, cl A denotes the closure of this subset. 

On the sets of all functions that are continuous on the whole T, we consider 
the topology of uniform convergence on T and the compact-open topology; for 
example, C(T,E) and Q oc (T, E). The first one is considered to be equipped with 
the norm || ■ \\q of the uniform convergence topology, fl denotes the family of 
functions uj G C(T,T) such that lim to(t) = 0. 

t— >-oo 

Here and below, for each summable function a of time, the integral J T a(t)dt is 
the limit Jj Q T j a(t)dt as T — > oo. The integral over an infinite interval, for example, 
over [T, oo), is interpreted in the same sense. 

Let us also consider a finite-dimensional Euclidean space U and a set-valued 
map U : T ~^ U. The set il of admissible controls is understood as the set of 
all Borel measurable selectors of the multi-valued map U. The topology on il is 
defined by virtue of the inclusion il C £j oc (T, U). 

A function a : T x E 1 x U — > E" is said to 

1) satisfy the Caratheodory conditions if a) the function a(-,y,u) : T — > E" is 
measurable for all (y, t,u) £ X x GrU, b) the function a(t, ■, ■) : E 1 x U(t) — > E" 
is continuous for all t 6 T. 
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2) be locally Lipshitz continuous if for each compact K £ (comp)(T x E) there 
exists a function L a K £ £,j oc (T,T) such that for all (t,x'), (t,x") e K,u £ U(t), 
the inequality \\a(t,x',u) — a{t, x",u)\\e" < ^ c kif)\\ x ' — x "\\e' holds. 

3) be integrally bounded (on each compact subset of T x E) if for each compact 
K £ (comp)(T x E) there exists a function M% £ £ i 1 oc (T,T) such that for all 
(t,x) £ K,u £ C/(t) we have \\a(t,x,u)\\ E '> < M^(t). 

4) satisfy the continuability condition on T if it satisfies the sublinear growth 
condition, i.e., if the function / is Lipshitz continuous such that the function 
L a K is independent of K and is integrally bounded (on each compact subset); 
see [33 1-4.6]. 



Here and below, we assume the following conditions hold: 

Condition (u) : U is a compact- valued map such that it is integrally bounded 
(on each compact subset of T) and GrUG B(T x U). 

Condition (fg) : the mappings / :TxXxU-> X,g : T x X x U — > R are 
locally Lipshitz continuous Caratheodory mappings that are integrally bounded 
(on each compact subset) and / satisfies the continuability condition. 

Let us consider the control system 

x = f(t,x,u), x(0) = 0, t £ T, x £ X, u £ U(t). (la) 



Now we can assign the solution ((Taj) to every u £ il. The solution is unique and it 
can be extended to the whole T. Let us denote it by (p[u]. The mapping if : il — > 
C; oc (T,X) is continuous. 

In what follows, we examine the problem of maximizing the objective functional 

lim Jt( u ) max; 

(lb) 



Jr(ti)= / ff(t, ¥>[«](*). «(*))#• 
Jo 



If there is no limit in (jlb[) . the optimality may be defined in diverse ways (for 
details, see [13] , [11] , [32] ) , generally, we will us the following one: 

Definition 1 A control u° £ il is called uniformly overtaking optimal if for each 
e £ R>o there exists T £ R>o such that Jj(u ) > Jt{u) — e holds for all u £ il, 
t £ [T,oo>. 

Note that in paper [32], this definition is referred to as uniformly catching up 
optimal control. In [211 Theorem 3.1], it is shown that the uniformly overtaking 
optimality is equivalent to the condition 



lim (j t (u°) - sup J t (u)) = 



which, in terms of [3T], says that u° is strongly agreeable. 

Note that for each uniformly overtaking optimal control u° £ il there exists a 
function uj° £ fl such that 

Jt(u°)>Jt(u)-uj°(T) Vu£il,T£T,t £ [T,oo>. (2) 
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2 On existence of uniformly overtaking optimal control 

To complete the main objective of this section, we need the following assumption: 
Condition (e) : there exists a function uj £ ft such that 

f g(t,tp[u](t),u(t))dt < u(T) Vu G Vl,T,t G T,T < r. 
■IT 

Note that to the best of author's knowledge, a one-sided condition like (e) was 
first proposed in paper [161 (778)]. As it was actually proved in [TBJ Subsect 5.1], 
instead of (e), it is enough to assume, for example, the stronger condition 

g(t,<p[u](t),u(t)) < l(t) Vu G il, T G T 

for some summable on T mapping I G £ 1 (T,R). 



2.1 The definition of the set il of generalized controls 

For each u G U, the symbol 5{u) denotes the probability measure concentrated at 
the point u. Let iln denote the family of all weakly measurable mappings fj, from 
[0,n] to the set of Radon probability measures over U such that /y™ q(t)(du) = 1 
for a. a. t G [0,n]. Let us equip this set with the topology of *-weak convergence. 
Then, the obtained topological space is a compact [341 IV. 3. 11], and the set il rl = 
{ u l[o,n] i u £ ^} is everywhere densely included in iln [341 IV. 3. 10] by the mapping 
it — y 5 o u. 

Now, let us introduce the set of all maps r? from T into the set of Radon 
probability measures over U such that »y[[o,n] £ ^« f° r every n G N; and let 
us denote it by il. To each n G N let projections 7r n : il — > il n be given by 
Kn(r]) = ??| [o.n] f° r a ll J? £ iX. Let us equip il with the weakest topology such that 
all projections are continuous. The set il is called the set of generalized controls. 

Let us assume that for the certain Euclidean space E a mapping a : Txi?xU-> 
(comp)(E) is given and the following condition is satisfied: 

Condition (a) : the mapping a : T x E x U — > (comp)(E) is a locally Lipshitz 
continuous integrally bounded Caratheodory mapping that satisfies the continua- 
bility condition. 

Let us fix the set S C E of initial values and the system for u G il: 

y = a(t,y(t),u(t)), 2/(0) = £ G S, £ G T, it G il. (3) 
It can also be generalized for 77 G il: 

y= I a{t,y(t),u)ri(t)(du), y(0) G S, teT.oeS. (4) 

JU(t) 

Each its local solution can be extended to the whole T. For every 77 G il, let us 
denote the family of all solutions y G C; oc (T, E) of system @ by QL[r]]. 
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2.2 The relaxed infinite-horizon optimal control problem. 

Similarly, we can consider the solution tp[rj\ G Q oc (T,X) of the Cauchy problem 
f(T,x(r),u)ri(t)(du), x(0) = Vn G 5, (5a) 

C7(t) 

the function T n> Jt(ji) = J[ T ] /[/(t) s(*> (*)i u ) v(t)(du) dt; and the problem of 
maximizing the functional 

lim J<r(tt) — > max . (5b) 



Proposition 1 Assume (u). Then, 

1) the space it is a compact, and <5(it) is everywhere dense in it; 

2) If (a) holds, then for a compact S G (comp)(E) the map 21 : iX C; oc (T , E) is 
continuous, and 2l[<5 oil] is everywhere dense in 2l[il] 6 (comp)(C; oc (T,ii')); 

5; // (fg) hold, then >p,J€ Q oc (T x it, R); 

4) If (fg), (e) /10W, f/ien t/iere is a uniformly overtaking optimal control vP G it /or 
the relaxed problem (I5ap (|5feD sitc/i t/iat 



lim sup / <;(*, ip[u] (t), u(t))dt = lim max Jxiv) = max lim Jt(v) = 

/• /■ (6) 

= lim J T (w°) = / / g(t,^[u°}(t),u)u°(du)dt, 

and all limits in © exist, although they can equal —00 

Proof. For the sake of brevity, let us denote II = YlneN an< ^ ^ us ec l m P it with 
Tikhonov topology. Let A : it — >• 77 be given by Z\(??) = (^«( , ?)) neN 101 all ^? G it- 
It is a homeomorphism by continuity of the maps n n and 7r n o A . 

Let n, G N, (n > fc). Then, the space itn is included in itj, by the mapping 

TTkiv) = >7|[Q,k] f° r all ^ £ Wri- By 5rj? o = 7T™ for all n, fc, i G N, (n > > i), we 
have the projective sequence of the topological spaces {it n , 7r^}; and we can define 
the inverse limit [181 III. 1.5], [171 2.5.1]. In our notation, we can write it in the 

form lim<- {it n , 7?^ } = Zi(il) C II. As shown above, A is a homeomorphism; hence, 
it is homeomorphous to A(SX). Now, by Kurosh Theorem [181 III. 1.13], the inverse 
limit Zi(il) of compacts itn is compact, and it is a compact too. Similarly, from 
[T71 4.2.5] and [34, IV. 3. 11] it follows that it is also metrizable. 

Repeating the reasonings without ~ or referring to |17l 3.4.11] and |17| 2.5.6] 

yields it lim{il„,7r£} = A(il) C II. 

For each n G N, let the mapping e n : il n — > it n be given by e n (u)(t) = (Sou)(t) = 
$u(t) f° r all i G [0, n], u G it n - Since for all n, k G N, n > k it holds that o = e n , 
we have the projective system {e n , ^ }• Passing to the inverse limit, we obtain the 
mapping : Z\(ii) — > A(li); from e„oir„ = iv n o S we have o A = A o <5, and 
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from il„ = cZe„(il„) ([34 ) we have zA(il) = de A (A{U)) = d(AoS)($X); now, by 
continuity of A~ x , we obtain il = d <5(il). 

The mapping 21 [77] is continuous by virtue of, for example, [331 Theorem 3.5.6]; 
the set 2l[rj](il) is compact as a continuous image of a compact. In what follows, 
is sufficient to use il = d 5{iX). 

Replacing a and the compact S with the mapping {(/,<?)} and the compact 
{(Oxi OR)}, we obtain the continuous dependence on rj for the maps ip, J. Now, by 
virtue of dJt (il) = dJt{6 oil) = J*(il), the condition (e) holds for r\ G il too, i.e., 
it holds that 

Mr}) < J T {n)+u{T) V?7 G it, T G T, t G [T, 00) , (7) 

then, 

lim sup J t (ri)< J T (r))+w{T) V?7Gii,TGT, 

t— >OQ 

passing to the lower limit as T — > 00, we obtain, for arbitrary rj G il, the existence 
of the limit lirm_ ) . 00 Jt (rj) (possibly infinite) . 

Then, for every teT, there exists an 774 G il such that 

9t t = max Jt (17) = J t (th) Vt 6 T. (8) 

Since (rjt)t£T is m the compact, for the certain unbounded increasing sequence 
(^fe)feeN G T and the certain u° G it, it is r/t k — » 5°. Let us also define 

!K = limsup9t t , S = liminf«H t , $R* = sup lim J t M, SH° = lim J t (u°). (9) 
Now, 

~ 

^t k =Jt k (vt k ) < JuM + "(U) Vi,fcGN(i<fc), 

passing to the upper limit as k — > 00 and then as i — > 00, we obtain $H < Jt ; (u°) + 
w(tj) and < Thus, for all T G T 

St < JR° P lim J t (w°) < sup lim J t ( V ) § JK* 

t— >oo ~ t— >oo 

T) Gil 



ja 

< sup J T (77) + w(T) = 9t T +w(T). 

Passing to the lower limit as T — > 00, we obtain 9t < 9t < 9t* < 9t, it remains to 
note that by virtue of dJt (il) = dJt(5 oil) = Jt (il), it holds that 

lim sup Jt(u) = lim max Jt (rj) = lim Dit = 9t° = 9t*. □ 

Remark 1 As it was shown in 1), for each generalized control there exists the 
sequence of controls from il that converges (in the topology il) to it. 

Remark 2 (turnpike property) Item 4) actually shows more. It shows that the uni- 
formly overtaking optimal control v° can be obtained as a limitary point of the 
sets argmax g y Jt{v) £ (comp) (il) as t — > 00. 
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Remark 3 As it was shown in 4), the limit 

/ / ff(*,£b7](*)X)'7( du ) d * = lim J T (rj) 

JT JU(t) T^oo 

is defined (though it may be infinite) for all r\ G it. 

Note that not only did the paper [16] prove the theorem of existence of an 
optimal solution based on the condition (e) but it also discussed the proof of such 
theorems based on the inverse limit. To the best of author's knowledge, there is 
only one paper [23] besides the previous one in the control theory that explicitly 
employs the notion of inverse limit. 

There are many existence theorems, for example, [9], [11] , [13] , [12] . The results 
obtained in Proposition [1] have much in common with paper [12] (in terms of [12] . 
the obtained u is strongly optimal) . Note that if the initial set it does not contain 
a uniformly overtaking optimal control, we may pass to Gamkrelidze controls by 
increasing the dimension of the set Uinm + 1 times. (For details of such bicompact 
extension, see [19], [12]). These controls also form a compact and the items 1)- 
3) of Proposition [I] hold from them; therefore, there always exists a uniformly 
overtaking optimal control among such finite-dimensional controls. 

As a corollary, we assume the uniformly overtaking optimal control u° to exist 
among the elements of it, and denote the trajectory that corresponds to u° by x°. 

We also keep the denotation u° = 5 o u°. 

We are also interested in the degree of closeness of various generalized controls 
for large t. Let w \JT x U — > T be an integrally bounded Caratheodory map. For 
all r 6 T and r\ e it, let us introduce 

£wM(t) = / / w(t,u)rj(t)(du)dt. 
■Jo JU{t) 

Let us denote by (Fin)(u ) the family of r\ 6 it such that t?|[t,oo) = ^°\[T,oc) f° r 
the certain T e T. Let us assume that £^,[5°] = 0, and for every r\ £ (Fin)(u°) 
from £„j[r;] (r) = for all r G T it follows that 77 equals u° a.e. on [0,r]. The set of 
such w is denoted by (Null)(u°). 

3 The necessary conditions of optimality 

3.1 Relations of the Maximum Principle 

Let the Hamilton-Pontryagin function H : X x Gr U x T x X R be given by 

H(x,t,u, X,ip) = ipf(t, x,uj + Xg(t,x,uj . 
Let us introduce the relations 

x(t) = f(t,x(t), «(t)); (10a) 
i>(t) G - d x H(x(t),t,u(t),\,ip(t)); (10b) 

sup H(x(t),t,p,X,ip(t)) = H(x(t),t,u(t),X,ip(t)); (10c) 
peu(t) 

x(0) = 0, ||V(0)||x + A = l. (10d) 
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It is easily seen that for each u G 11, for each initial condition, system (|10a[) - 
(|10b[) has a local solution, and each solution of these relations can be extended to 
the whole T. 

Let us denote by 2J the family of all solutions (x,u,X,ip) G C; oc (T, X) x 11 x 
[0,1] x Q oc (T,X) of system ([TPa|l - ((T0b|l . (fi"0d|) on T, and let us denote by 3 the 
set of solutions from 2) for which (|10c[) also holds a.e. on T. 



Let us introduce such conditions for generalized controls; namely, under initial 
condition (|10d|) let us consider 



*(*)=/ f(t,x(t),u)ri(t)(du); (11a) 

JU{t) 

1>(t)€- [ d x H(x(t),t,u,\,i>{t))r)(t)(du); (lib) 
Ju(t) 



sup H(x(t),t,p,X,ip(t)) = / n(x(t),t,u, X,ip(t))r)(t)(du). (11c) 
p€U(t) JU(t) 



Similarly, for each r\ g 11 for each initial condition, system (|lla[) - (|llb[) has a local 
solution that can be extended to the whole T. 

Let us denote by 2) the family of all solutions (x,r/,X,ip) G Q oc (T,X) x 11 x 
[0,1] x C; oc (T,X) of system (|10d[l - (|llbp . Let us also introduce 3, the family of 
(x,r],X,ip) G 2) such that (|llc[) also holds a.e. on T. 

Let us note that for every 77 £ 11, the family of all solutions (x,ri,X,ip) G 
2) of system (|10d[) - (|llap on T for given control 77 is compact by virtue of [331 
Theorem 3.4.2]. Moreover, this compact-valued map is upper semicontinuous in 77. 



Indeed, the right-hand side of (|lla[) - (|llb[) is convex and integrally bounded, upper 
semicontinuous in 77, and it is measurable for each fixed x,ip; therefore, it has a 
measurable selector ( (331 Lemm 2.3.11]); moreover, all local solutions of (|lla[) - 



(lllb[) can be extended to the whole T. Since all the conditions of [33, Theorem 
3.5.6] are satisfied, the mapping is upper semicontinuous. Therefore, 2) and 3 are 
compact, as the graphs of this mapping on the compact subdomain of its domain. 
Note that by [141 Theorem 2.7.5] always holds the inclusion: 

d x H(x,t,u,X,i})r)(t){du) c/ d x U(x,t,u, X,ip)r](t)(du). (12) 
JU(t) JU{t) 



3.2 The necessity of the Maximum Principle 

Theorem 1 Assume conditions (u),(fg). For each uniformly overtaking optimal 
pair {x°,u°) G C(T,X) x 11 for problem $Ta} - ([76)) . there exist X° G [0,1], ip° G 
C(T,X) such that the relations of the Maximum Principle (1 10a^ - <\10d^ hold; i.e., 
(x°,u ,X ,i,°)€3. 

Proof. 

Let us fix a certain unbounded monotonically increasing sequence (T ra ) ng N ^ 
T N . Let us also consider an arbitrary sequence (7n)neN £ T N that converges to 
zero with the property cj (T n )/7„ — » 0, where the function io° was taken from ([2]). 

For example, ^y n = \J Lj°(T n ) will suffice. 
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Fix awe (Null)(u ). For each n G N let us consider the problem 



Jt„(v) - ~/n£w[v]{Tn) = / / g(t,ip[j]](t),u)ri(t)(du)dt - 7n£ w [r?] ( T «) ->• max. 
Jo Ju{t) 

Here, the functional is bounded from above by the number J Tn (u) + uj°(T n ), 
therefore, it has the supremum. Every summand continuously depends on 77, which 
covers the compact H; therefore, there is an optimal solution for this problem in il; 
let us denote one of them by (x n , if 1 ). 

Let the function H T „ :XxGr(7xTxX->Rbe given by 

H Tn (x,t,u, X,tp) = H(x,t,u, A,i/>) — y n w(t,u). 

Then, by the Clarke form [141 Theorem 5.2.1] of the Pontryagin Maximum Prin- 
ciple, there exists (A", ip n ) G T x C([0,n],X) such that relation (|10d|) and the 
transversality condition at the free endpoint ip n (T n ) = hold, and 



sup H Tn (x n (t),t,p,\ n ,Tp n (t)) = / Hr n {x n (t),t,u,X n ,^ n (t)) V n (t)(du), (13) 
peu(t) Ju{t) 



^ n (t) g -a* f n Tn {x n (t),t,u,x n ,^ n (t)) v n (t)(du) 

Ju(t) 



IU{t) 

also hold for a.a. t G [0,r„]. By (x n , if, X n , •0") G T x C([0,n],X) satisfy the 

relations (|10d|) - (|llb|) . (fT3j) a. e. on the [0,r„]. 

Let us extend the (x n , r) n , A™, ip n ) to [r n ,oo) by the generalized control m |[ Tn 00 \. 
Then, rf 1 G (Fin)(u°). Let us denote by 3" the set of (x,u,X,ip) that satisfy 
relations (|10d[) - (|llb[) a. e. on T, satisfy relation (|13p a. e. on [0,r n ), and possess 
the property u°|[ Tni00 ) = V n \[ Tn ,oo)- Now we have (x n , r) n , A n , V» n ) G 3™ for every 
n G N. 

Let us note that all 3" are closed and, since these sets are contained in the 
compact 2), these sets are also compact. Hence, the sequence (x n , rf 1 , A™, V> n )neN 
has the limit point (a; 00 , rj , A , ip ) G 2). Passing, if necessary, to a subsequence, 
we may assume that it is the limit of the sequence itself. 

For a fixed x, the set of u G U(t) that realize the maximum in (|13[) has a 
measurable selector by virtue of [15, Theorem 3.7]. By [331 Lemm 2.3.11], it exists 
if we put an arbitrary continuous function x into H. Besides, since relation (|13[) also 
depends on x, if) and on the parameters 7 and A upper semicontinuously, and all the 
relations are integrally bounded on bounded sets; by virtue of [331 Theorem 3.5.6], 
on each finite interval for the funnels of solutions of (|10a[) - (|10b[) that satisfy (|13p , 
we have upper semicontinuity by 7, A. In particular, for 7„ — > 0, A™ -> A , we 
obtain the fact that the upper limit of the compacts 3™ is included in 3- Hence, 

(x m , v °,x°,^)el 

On the other side, by w G (Null)(u ) and by optimality of rf 1 , u° for their 
problems, we obtain 

Jr n (r] n ) - Jn£w[r} n ](T n ) > Jr n (u°) > J T „(ri n ) - La°(T n ) 

therefore, we have r i n £w\rf l \{T n ) < w (r n ). By virtue of u°|[ T „ i00 ) = 77™|[t„,oo): we 
obtain 

•Cwfo n ](T) <w°(t„)/t„ VrGT. (14) 
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For each r G T, passing to the limit as n — > oo, we obtain that -Cu^t? ] < 0; i.e., 
£hj[t? ](t) = for all r € T. Since w G (Null)(u°), we have 77° = u° a.e. on T, hence 
x 00 = x° and (x°, u°, A , ip°) G 3. Moreover, from (HU), we have |]£«;[?7 n ]]|c -> 0. 

□ 

We have additionally proved that 

Remark 4 Under conditions (u),(fg), for each optimal pair (x°,u°) G 3E x H for 
problem (|lb|) . for each weight w G (Null)(u°), for each unbounded increasing 
sequence (r n ) ne N S T N , we have constructed the sequence (x n , 77™, A™, i/i")„ e n G 
2) N that possesses the following properties: 

1) This sequence (as a sequence from C; oc (T,X) xHxTx Cj oc (T,X)) converges 
to the certain (x°,v°, A ,t/>°) G 3; 

2) H£™(rf)||c^0; 

3) Jt n (v) — Jt n ( u ) ~^ 0; an d tp n (tn) = for each n G N, where (t n ) n £~N is a certain 
subsequence of (T n )n£N G T N . 



3.3 The simplest condition of transversality 

However, the relations of the Maximum Principle are incomplete, since (|10a[) - 
(|10d[) do not contain a condition on the right endpoint. There are several variants 
of such additional conditions (for details, see ;4, Sect. 6, 12], [29]); in this paper we 
investigate the modifications of the condition 

lim iji(t) = 0. (15a) 

t— >oo 

Let us formulate the propositions in terms of the stability of ip such that a 
condition would be necessary. 

Condition (1/)): There exists a weig ht w G (Null)(u°) such that for every so- 
lution (x° ,v° , \° ,ip°) G 3, the Lagrange multiplier ip° is stable under £, w — small 
perturbations of system (|10a[) - (|10b[) ; i.e., for every s G R>o, there exist a number 
8 G R>o and a neighborhood T C C/ oc (T, X) x iix [0, 1] x C; oc (T, X) of the solution 
(x , S°,A , 4> ) such that for every solution (x,rj, \,ip) G T n 2) from H-CtuHllc < ^ 
it follows that — i/j°\\c < £ - 

Proposition 2 Assume conditions (u), (fg) hold. For each uniformly overtaking opti- 
mal pair (x , u ) G C"(T, X) xil satisfying (t/j), /or eac/i unbounded increasing sequence, 
( r n)neN £ T N t/iere exists (x° ,u° , \° ,ip°) G 3 suc/i that 

liminf ||i/>°(t„)||x = (15b) 

Proof. Let us choose the certain e G R>o, and let us take Tc5) and 8 G R>o from 
condition (ip); by Remark [5] there exists AT G N such that for n G N, n > N, it 
is (a;", 77™, A n , V>") G T, ||£-47? n ]|| c < 5; now, condition (V>) also yields ||V> n (T„) - 
V^^rOHx < e; but i> {r n ) = 0; whence ||^°(T n )||x < £ for all 77, G N, n > AT. Since 
e G R>o was arbitrary, we have shown (|15b|) . □ 



12 



Dmitry Khlopin 



Note that by linearity of (|10b[l , the stability of the variable ip implies its bound- 
edness. Therefore, the proved proposition is useless for unbounded adjoint vari- 
able ip. 

Note that, as it follows from [321 Example 5.1], for a uniformly overtaking 
optimal control, there can be no (x°, u° , A , ip°) G 3 that satisfies stronger condi- 
tion (|15a|l instead of (|15b[) . On the other side, 

Remark 5 Assume the functions L^ K ,L 9 K are independent of a compact K, and 
the mapping T h-> 

L 9 K {T)eho,n L K (t)dt ig summable on T (j27[ Hypotesis 3.1 (iv)]); 
therefore, the total variation of ip is a fortiori bounded. Then, (tp) holds and, 
moreover, (fi~5b)) impl ies (|15a(l . 

The even more strong conditions used for proving the Maximum Principle 
can be seen, for example, in [361 (A3)] (the Lipshitz constants were required to 
decrease exponentially with time) . Naturally, the propositions proved there for the 
condition are also covered by proposition [2j 

One of the most general conditions on (|15a[) was shown in [28] . For a control 
problem without phase restrictions, the transversality condition from 28, Theorem 
6.1] follows from Proposition^ and [281 Lemm 3.1], or from Remark[5]and condition 
[281 (C3)]. The Remark H automatically yields [2H Theorem 8.1]. 



4 The necessity and the stability 

The objective at hand is to choose the weight w° G (Null)(u ) such that condi- 
tion (ip) would follow from a variety of (nonasymptotic) Lyapunov stability of ip. 



4.1 On weight w° 

Assume conditions (u), (a) hold. In what follows, assume 3 = E. Then, for every 
position (t*, j*)£Txl? there exists the unique solution y of the equation 



that can be extended to the whole time interval T. It (as an element of 21(5°) C 
Ci oc (T,E)) continuously depends on (r*,y*) G T x E. Let us denote its initial 
position y°(0) by x(r*,y*). 

Proposition 3 Assume (u), (a) hold. Let the compact-valued map G : T ~-> E be 
bounded on each compact set, and let Gr G be closed. 

Then, there exists w € (Null)(u°), such that for arbitrary r\ €.01, T € T for every 
y G 2l[?7] from Gr j/|[q,t] C Gr G it follows that 



y = a{t,y(t),u°{t)), y(r*) = y* , r* G T 



(16) 



||x(T,»(T))-y(0)|| B <i^o[!7](T) 



Vr G [0,T]. 



Proof. 
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Fix an n £ N. By continuability, for each {r*,y*) £ GrGU 0>n j, there exists the 
position k(t* ,y*)\ by virtue of the theorem of continuous dependence on initial 
conditions, this mapping is continuous; hence, the image 



G A 



„ £ { e e Gr y\ l0tn] \Vy £ (r*,y(r*)) e Gr G\ l0 , n] } 

is closed; by the continuability, this set is bounded and, therefore, compact. There- 
fore, on this set, the function a(t,y,u°(t)) is Lipshitz continuous with respect to y 

for the certain Lipshitz constant L n = £ £ J 1 oc (T,T). For all t £ [0,n], de- 

fine M n (i) = Jj t j L n {r)d,T. Note that this function is absolutely continuous and 
monotonically nondecreasing. 

Fix n £ N; for all t £ [n — 1, ri), u € U, let us consider a number 

R(t,u) = sup \\a(t,y,u) - a(t,y,u°(t))\\ E . 
yeG n 

Note that the norm inside is a mapping that is continuous with respect to y 
and u, and y assumes values from the compact set; now, for every u £ U 
by [151 Theorem 3.7] the supremum reaches the maximum for the certain function 
y-max [u ] £ C l {[n,n-l),G n )- Hence, R{t,u) is measurable with respect to t for each 
u £ U. 

Fix a t 6 [n — l,n); for each sufficiently small neighborhood Y C U(i), by 
continuity of a(t, ■, ■) on compact G n x clT, there exists a function u* £ ft, for 
which 

| \ \a{t,y,u) - a(t,y,u°(t))\ \ - \\a(t,y,u") - a(t,y,u°(t)) || | < w * (|u7Z^77T|) ( 17 ) 

holds for every y £ G n ,u ,u" £ T (V ^ u"). Without loss of generality, as- 
sume R(t, u) < R(t, it"). Now, by definition, R(t, u) > | ja(t, y, u) — a(t, y, 1 1 , 

and, substituting y = y m ax {u")(t) into (HTJ), we obtain < R(t,u") - R(t,u') < 
— u"\\); i.e., R is continuous with respect to the variable u on each suffi- 
ciently small neighborhood Y C U(t); therefore on U{t) and GrU\^ n _ l n ^ too. Thus, 
the function R : GrU\^ n _ 1 ^ — > T is a Caratheodory function. 

Let us note that by considering all n £ N, we define the Caratheodory func- 
tion 7? on the whole GrU. Moreover, by construction, R(t,u°(t)) = 0. Hence, it is 
correct to define w° £ (Null)(u°) by the rule 

w°{t,u) = \\u-u°(t)\\ + e Mn(t) R{t,u) Vn £ N, (t, u) £ Gr U\ [n _^ n) . 

Consider arbitrary n £ N, r* £ [0,n], and (r*, 2/1), (r, y%) £ G n - For the solu- 
tions j/i, 2/2 6 Sip ] of equation (|16[) . for the initial conditions yi(r*) = y* , we have 
C r 2/il[o,n] C G n - Let us introduce functions 

r(t) = yi(t)-y 2 (t), W+(t) = e M ^\\r(t)\\ E \/t e [0,n]. 

By Lipshitz continuity of the right-hand side of (|16[) we obtain > 
-L n (t)\\r(t)\\ E , and 

= 2L n (t)Wl(t) + 2e 2M ^r(t)r(t) > 2L n (t)Wl(t) - 2L„(t)W^(t) = 0. 
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Thus, the function W+ is nondecreasing, and finally for all (r, y*), (r, y 2 ) S G n we 
have 

\\x(T,yt) - x(r,y* 2 )\\E = W+(0) < W+{r) = e M ^\\y{ - y* 2 \\ E . (18) 

Assume the 77 € it, y 6 2l[r?], T G T satisfy Gr 2/|[q,t] C GrG. Fix arbitrary 
n £ N and ti,t 2 £ [0,T] H [n - l,n), r\ < t 2 . There exists the solution y° G 2lfu°] 
that satisfies the condition y°(ri) = y(ri); let us also define 

r = y°(t)-y(t), W-(t) = e~ M ^\\r{t)\\ E Vte[n,r 2 ]. 
By construction of G n , we have Gr y\[ Tl , T2 ] , Gry°\^ Tl ^ C G n . Now, 

= 2e' 2M ^r(t)r(t) - 2L n (t)Wl(t) = 

2e- 2M ^r(t)(y°(t)-a(t, y (t),u°(t)) + a(t,y(t),u° (t))~y(t)) - 2L n (t)Wl(t) < 

2e -3M " w ||r(t)||B / R(t,u) v {t){du) + 2L n (t)wl(t) - 2L n (t)Wl(t) < 
Ju(t) 

2e - M ^W-(t) [ R(t,u)r,(t)(du) < 2 e - 2M °t%_(t) ^M , 
■ 'u(t) "* 

Since function W_ is nonnegative, for a. a. t 6 {( 6 [n, t 2 ] | ^ 0} we obtain 

dW- (t) < c -2M n (t) d£ w o[rj\(t) < e -2M„(T 1 )^ H 4#) / 1q n 
dt ~ dt dt ^ ' 

This inequality is trivial for [Ti,ra] 3 t < sup{t 6 [ti,t 2 ] | W_(i) = 0}; whence, 
\\x(T2,y°(T2)) - x(T2,y(T2j)\\E < e " (T2) I \y°(T 2 ) - 2/(r 2 ) | \e = 

™^) W _ {T2) <§» e2 M„(r 2 )- 2 M„(. l)(£rooW(r2) _ £u)oW(Tl)) . 

But x(r 2 ,y (r 2 )) = j/°(0) = x(n, 2/°(n)) = x(n, 2/(n)), hence, we have 

lk(r 2 ,y(r 2 ))-x(r 1 ,y(r 1 ))|| £ <e 2M "^)- 2M "^)(£ w oH(r 2 )-£ t „oM(r 1 )). (20) 

Fix arbitrary t £ [0,T]. For each e £ R>o we can split interval [0, t) into the 
intervals of the form \t',t") such that M„(r") - M n (r') = Jj T , r „. L n {t)dt < e and 

[r',r") C [n — l,n) for the certain e N. But, (1201) holds for every interval, i.e., 

\\x{r",y{r")) - x{r' ,y{r'))\\ E < e 2e (£ M oM(r") - £ w o [„] (t')) • 

Summing for all intervals, by x(0,jy(0)) = y(0) and by the triangle inequality, 
we obtain \\x(t,y(t)) — y(0)\\ E < e 2e 2, w a[rj\(t) for every t £ [0,T]. Arbitrariness of 
e £ R>o completes the proof of the proposition. □ 
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4.2 The partial Lyapunov stability 

Assume E can be represented in the form E = E p x E q for some finite-dimensional 
Euclidean subspaces E p and E q . Let us denote the projections of the map a to the 
subspaces E p and E q by b and c, respectively. Now, the system ((3} can be written 
in the form 

p = b(t,p,q,u), q = c(t,p,q,u), (p, q) (0) = £ £ £, «6£/(t); (21) 

Then, it is possible to say that for all 77 6 it, the set %L[rj\ contains pairs of functions 
(p,q) € Q oc (T ; S p ) x C; oc (T, £? 9 ). For every £ e E, let us denote by y° = (p°, g°) £ 
21 [u°] the unique solution of (|16|) for t* = 0,y* = f. 

Definition 2 Consider a closed set Go C E and ^ £ Go. We say that the solution j/° 
0/ equation (|_?6p /las Lyapunov stable component p^ in domain Gq if for each £ 6 R>o 
there exists 5(e,y) £ R>o such that for each £ £ Go from ||£' — £||,e < 8(e,y) it 
follows that ||p°/(s) -pI(s)\\e < £ for all s £ T. 

Proposition 4 Assume (u),(a) holds. Suppose there is a closed set Go C E and a 
compact Kq £ (cornp)(Go) such that for each £ £ Kq the solution y^ of equation (I i pp 
/las Lyapunov stable component p® in Go . 

Then, for each s £ R>o ; i/iere exists a number 8 £ R>o such that for alln £ il, 3/ = 
£ SIM / rom 2/(0) G ^0, l^o [v]\\c < 5, ^d x{t,y(t)) £ G /or a/Z i £ T, it 
follows that ||p-p° (0 )llc < £■ 

Proof. Consider a compact A> = {£ £ Go | 3£o £ Kq ||£ — £o||b < !}• To 

each f £ T, let us assign the set G(t) = | r/ £ H, y £ 2l[r;],j/(0) £ A">}. The 

obtained map G is compact-valued and continuous; in particular, its graph is 
closed. Now we can use Proposition [3] for the multi-valued map G and fix the 
weight w° £ (Null)(u°) which exists by this Proposition. 
Define 

M(£',£") = sup||$(t) -p°^(t)\\E p G T U {+00} V£',£" £ K>. 
teT 

For all £ £ Ao, the stability of the component p® implies that the map M is finite 
and continuous at the point £ A> x A>. 

Fix an e £ R>o; choose for every £ £ Ao its 5(e/2, £ (0, 1/2]; now, we have 
also chosen the <5(e/2, 1/^)— neighborhood of the point (£,£) (in A> x A>). From 
the obtained cover of the diagonal A of the set Ao x Kq, let us select a finite 
subcover; it induces certain open neighborhood Y of the diagonal A. Let 8{Kq) be 
the minimum distance from the diagonal A to the boundary of the neighborhood T. 
Now, for all £' £ A>, £ £ A from |]£'-f||i5 < <5(A ) it follows that (£',£) £ T; i.e., 
for some £" £ Kq we have M(f,£"), M{1" , £') < e/2, whence M(£,£') < e. Thus, 

<«(#(>)) (l|P«'-P?)llc<e) V£ £ Ao , £ A> . (22) 

Suppose the rj £ it, y = (j>, 5) € S^r/] satisfy L^o [??](/;) < 5, £i(i) = x(t,y(t)) £ Go 
for all f £ T. For Kq C A> = G(0), the definition 

T =sup{r£T|Gr£i| [0 ,t] C A> Vt £ [0, T» £ T U {+00} 



16 



Dmitry Khlopin 



is correct, although To can be infinite. Hence, we have Gry\m^ C GrG for all 
t G [0,T). Now, from Proposition [3l we obtain 

||6(t) - tf(0)|b = \Ht,y(t)) - y(0)|| B < £ w o[7?](t) < 5(K ) Vt G [0,T ). (23) 

For every t G [0,T >, let us substitute $ = y(0),£,' = ^i(i) G -R"> in ([22]); from the 
equality = we obtain \ \p(t) — Py(o) (*) I l-E" < £ f° r a ^ IP> ^b). To conclude 

the proof, it remains to prove that To = oo. 

Suppose To G T; by construction of To, for each r G (To,oo), we have 
Gr^i| (To r] £GrK>- but GrCi G G . Then, p^ 1 (T ),G \K > ) = 0, and, in partic- 
ular, by construction of _K>, we have ||£i(To)— J/(0)||e > 1. However, passing to the 
limit in ([23]) yields ||£i(T ) - y(0)\\ E < S(K ) < 1/2. The acquired contradiction 
proves that To = oo. □ 



4.3 The necessity of the transversality condition (|15bp 

Everywhere further, we assume the following condition holds: 

Condition (<9) : for the maps (t,x) G TxXxU-> f(t, x, u) G X and (t, x) G T x 
XxU-> g{t,%, u) G R on their respective domains, there exist partial derivatives 
in x that are integrally bounded (on each compact) locally Lipshitz continuous 
Caratheodory maps. 

Under this condition, the set dxWix^t) ,t, u° (t) , \, 4>(t)) is also a single-element 
set, therefore system (|10al) - (|10b[) can be rewritten for u = u° in the form 

Ht) = —^(x(t),t,u°(t),\,ip(t)), (24a) 

x(t) = f(t,x(t),u°(t)), (24b) 
A = 0. (24c) 

Corollary 1 Assume conditions (u),(fg),(9) hold. Let the pair (x°,ti°) G 
C/ oc (T,X) x it fee uniformly overtaking optimal for problem (| jqp - (| ifcj) . If for each 

solution (ip° , x° , A ) of system ( |£^ap - p^cp initial conditions from Kq = D x 

[0, 1] x {Ox} the component rjp is partially Lyapunov stable in Go = X x [0, 1] x X. 
Then, the result of Proposition [£] holds. 

Proof. In (121[) . it is sufficient to define E p = X, E q = X x R and to take tp 
and (a;, A) for p and q = (51,52), and to take for b and c the right-hand sides 
of (|24a[) and (I24b[) - (|24c[) . respectively. Now Proposition [3] guarantees (i/>), i.e., all 
conditions of Proposition [2] are met. This proves Corollary [T] □ 

For Go, we can take the image {x(t,7/;(f),a;(t), A) | (2;, u, A, i/j) G 2),i G T}. 

Using Proposition [3] and Remark [3] instead of Proposition [3] and Corollary [T] in 
this proof, we obtain 

Remark 6 Under conditions (u), (fg), (d), for each uniformly overtaking optimal 
pair (x°,u°) G G; oc (T,X) x il for problem (jlbjl . for each unbounded increasing 
sequence (r„)„ e N G T N , we have constructed the sequence (x n , rf 1 , A™, ■i/'™)rieN £ 
2) N such that 
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1) this sequence (as a sequence from C/ oc (T,X) xllxTx C/ oc (T,X)) converges 
to a certain (a; , vP, X°,ip°) £ 3; 

2) the graphs Gr(x n , A™, ip n ) of its elements are contained within the thinning 
funnels of solutions of system (|24ap - (|24c[k i.e., for a sequence (<5„)„ e N £ R->o 
that tends to 0, we have 

x{t, O™, x 11 , A™)) £ (V)°(0), 0, A ) + <5„D x <5„D x [-5 n , S n ] V(6T,n£N. 

3) Jt n (v) — Jtn( u °) ~~ ^ 0, and ip n (t n ) = for each n £ N, where (tn) n eN is a certain 
subsequence of (r„)„ e N £ T N . 



4.4 Modifications of transversality condition (|15b|) 

In certain cases, if the Lagrange multiplier ip is not stable, but we know that 
certain components of the vector variable ip are stable, or we know the rate of its 
growth. Then we can try to select the mapping A* : T — > L, may help to modify 
condition (|15b|l . and use the condition 

liminf ||^ (iU*(t)|| x = (25a) 

t— J-oo 

for certain map A* :Th>L. 

Here are the examples of such maps A*: one that maps the unity matrix 
A*(t) = 1l; some "scalar" multiplier A*(t) = r(t)li,; a mapping A*(-) = D with 
the diagonal matrix D; the condition ip(t)x(t) — > 0, which is often used as the 
sufficient condition, can also be reduced to this form. 

Let us assume that for all 77 £ it, £ £ X, we have chosen the measurable mapping 

: T -> L. Assume A|(0) = 1 L for all 77 £ it, £ £ X. Define A* = A%° x . 

Condition (ipA): There exists a weight w £ (Null)(u°) such that for every 
solution (x°,u ,\°,ip ) £ 3 for each e £ R>o there exist a number 5 £ R>o and a 
neighborhood T C C; oc (T,X) xitx [0, 1] x Q oc (T, X) of the solution (x°,u°, X°,ip°) 
such that for every solutions 2 = (x,T],X,ip) £ Tn2), from ||£uj[?7]||c < 5 it follows 
that ||^ (0) -V°^0xllc<e- 

Proposition 5 Assume conditions (u),(fg) hold. For each uniformly overtaking op- 
timal pair (x°,u°) £ C(T,X) x it satisfying (ipA), for each unbounded increasing 
sequence (r n ) ne N £ T N there exists (x° , u° , A , ip°) £ 3 such that 

liminf |^ (r„)yl*(r n )||x =0. (25b) 

n— ¥00 

hold. 

The only differences between the proof of this Proposition and Proposition [2] 
are the facts that the references to (ip) are replaced with references to (ipA) and 
the factors Aj,A* are added to the inequalities of the last strings. 

Similarly, we can formulate an analogue to Corollary [1] for this condition: if it 
is possible to choose matrix maps such that the product 4>A™^ is the solution of 
an equation 

^ = b(t,p(t),m,<t),\,u(t)) (26) 
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for each u £ il for each solution z = (ip,x,X) of system (|10b|) . f)10ap . ()24c|) with 
initial conditions z(0) € E qi then the corresponding stability of this component p 
in system pS ]) . (|24ap ^ (|24cp implies the result of Corollary Q] (see [24]). 

The simplest way to account for the a priori information on stability or for 

asymptotic estimates of ip and its components is to take A*(t) = e~ At lL, where A 
is greater than or equal to all Lyapunov's exponents of the variable ip. In particular, 

in [281 Example 10.2], the use of A*(t) = e — in such condition (in contrast to 
the standard condition) selects the single extremal. 



5 Cauchy formula for adjoint variable 

In the papers [1,2,3,4,5 , Aseev and Kryazhimskii have proposed and proven the 
analytic expression for the values of the adjoint variables. This version of the 
normal form of the Maximum Principle holds with the explicitly specified adjoint 
variable providing a complete set of necessary optimality conditions; moreover, the 
solution of this form of Maximum Principle is uniquely determined by the optimal 
control. This approach generalizes (see [H Sect. 16], [7]) a number of transversality 
conditions; in particular, it is more general than the conditions that were obtained 
for linear systems in [Sj. 

It turns out that if the function A* is fundamental matrix of linearized sys- 
tem along the optimal trajectory, then, condition (|25a[) automatically yields this 
explicit representation for the adjoint variable. 

Let us simplify Proposition [5] for such A* to weaken the requirements of [H 
Theorem 12.1], 7, Theorem Theorem 2], and their corollaries. 



5.1 The case of dominating discount 

Let the pair (x°,u°) £ C/ oc (T,X) x il be uniformly overtaking optimal for prob- 
lem (|la[) - (|lb[) . Along with it, let us consider the solution A* of the Cauchy problem 

d _A^ = a f (t, x y,u\t)) 

at ox 

Likewise, for each (eX, let us denote by x^ the solution of (|10a|) for the initial 
condition 3^(0) = (eX; let us also consider A^r, the solution of the matrix Cauchy 
problem 

d A^ = mM ym AAtl M0) = lL v$ex . 

For each T £ T, let us consider 

Proposition 6 Assume conditions (u), (fg), (<9). Let the pair (x°,u°) £ C; oc (T,X) x 
it be uniformly overtaking optimal for problem dial) (!_?&[) . Let the map Jo be bounded 
and let 

lim||I € - folic =0. 
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Let J* G X be a partial limit (the limit of a subsequence) of Iq(t) as r — > oo. 

Then, there exists a solution (x°, u° , A , ip°) G 3 of all relations of the Maximum 

Principle {\10a\ - ^10dfy satisfying the transversality condition (|25a|) . Moreover, A = 

- — > 0, and ip G C/ oc (T,X) defined by the following rule: 

1 + IM*llx 

j,°(T) t X °(h - f Mt,At),u°(t)) Mt)d \ A -i {T) \/T G T. (27) 
• ' o 

Proof. For each u G 11 and z = ¥?[«], let us introduce a matrix function A u that 
is the solution of the matrix equation 

A u {t) = d/MW> M W) A u^ j = 1l (2g) 

Now, for each solution (z,u, \ u ,ip u ) G 2) from (|llb[) it follows that 

j(fi")(t) = _ x u dg{t,z{t),u{t)) AU{f ^ (2g) 

For 

A , - , . .A dg(t,q2,u) 
E p = X, 6(f,p, (51,92,93),") = -53 -R 5i, 

ax (30) 
fi, = LxXxR, c(i,p, (51,52,53),") = ( df{t d q x 2,U \ i,f(t 1 q 2 ,u),0) 

the system (gTJ becomes system (f29|) . (f28l) . (flOa)) . (f23c|) : now, for u = u° , 

= _ r dg(t,z u°(t)) ^ = df(t,z u°{t)) . = z u o {t)) = Q (31) 
ox ox 

Solving this system, we obtain 

r(t) = r(0), z{t) = x m (t), B{t) = A m {t)B(Q), 

p(t)=p(0)-r(0)I zm (t)B(0). (32) 

Let Go = X x L x X x [0,1]. We claim the partial Lyapunov stability of the 
component p in G for B(0) = l L ,z(0) = 0,r(0) G [0,1], p(0) G D. Indeed, by ([32]) 
it remains to verify that 1^ is continuous at the point £ = 0, and Iq is bounded; 
both hold by assumptions. Therefore, by Proposition |H for the certain weight 
w G (Null)(u a ), the component p is stable for £, w o— small perturbations of the 
control u . This provides condition (ipA) for A u that were defined as we did. 

By condition, 7* is a partial limit; hence, there exists an unbounded increasing 
sequence (T n ) n eN £ T N with property 7o(Tn) — > I*- Now, by Proposition [5l there 
exists (a;°,it°, A°,i/> ) G 3 with properties (|25b|) and (f25a|l . 

Substituting z(0) = 0, r(0) = A , A (0) = 1 L , andp(0) = V°(0) into fl32} yields 

(V ^»)(r) = (v A )(T)=p(r) = ^ (o)-A°/ (r) vtgt. (33) 

Now, substituting T = r n and passing to the lower limit, from (|25ap . we obtain 
= ip (0) — A /*; therefore, from (|33|) and (|10d|) respectively, we have 

^(T)^(r) = A (/»-J (T)), A°= 1+ * >0. 
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Using the inverse matrix for A* (T) , we obtain (|27l) . □ 
Let us note that if 7* is independent of choice of the subsequence (r n )„ 6 isf, we 
automatically obtain the stronger transversality condition 



lim i/j(t)A*(t) = 0. (34) 



Moreover, since for different (x°, it , A), solutions of (|29|) differ by a constant, for all 
(x , u° , \,i>) G 2), the products ipAo tend to a finite limit as t — > oo. If (|27|) holds, 
then this limit is equal to zero. Hence, to every (x°,u°, A) there corresponds at most 
one 4>°, for which relations (fT0a) -(fT0c |) . (|3l)) hold; now, from f)10d|) and ([32]) we 
can reconstruct A uniquely. Thus there exists the unique solution (a: , u°, A, Tp) € 3 
that satisfies condition (|34p , and the following theorem is proved. 



Theorem 2 Assume conditions (u), (fg), (<9) hold. Let the pair (x°,u°) G 
C; oc (T,X) x SI be uniformly overtaking optimal for problem (I iap - (|ife[) . and let the 
limit 

dg(t,x°(t),u°(t)) 



lim h{t)= — k 21 — —AJt)dt£~R 

t-s-oo,£-s-0x Jt " x 

be well- defined and finite. 

Then, there exists the unique solution (x°, u°, A , i/>°) G 3 of all relations of the 
Maximum Principle (| 10a$ - (| lOdfl satisfying the transversality condition Q3^p . Moreover, 
accurate to the positive factor, we can assume 

X°±l, V°(T)=/ d9{t ' X °^ U ° {t)) A^t)dtA-\T) VTGT. (35) 

J[T,oo) " x 

From conditions of Theorem 2], [2 Theorem 12.1],0 Theorem 1],0 Theo- 
rem 1] it follows that for some a, f3 G R>o and for all admissible controls u, all 
trajectories x, and all fundamental matrices A u , the inequality 



i 9g(t,3;(t),M(t)) j 
dx 



\A u (t)\\</3e~ at VtGT (36) 



holds. This is stronger than the conditions of Theorem 2. Informally, the require- 
ments of [3 Theorem l],[5l Theorem 2], [4, Theorem 12.1], [7, Theorem 1] boil 
down to the need for uniform exponential Lyapunov stability of the product ipA 
along all trajectories of the system dJ), while Lyapunov stability of the product 
ip° 'Aq along the optimal solution of the initial control problem is sufficient for The- 
orem 2. On the other side, the condition (|36[) can be verified by calculating the 
Lyapunov exponents of the system of the Maximum Principle, see [H Sect. 12], 
Sect. 3], [3 Sect. 5]. 



5.2 The general case 

Let us base on the start of the proof of Proposition^ and let us use not Corollary[T] 
but Remark 21 

Proposition 7 Assume conditions (u),(fg),(9) hold. Let the pair (x°,u°) G 
C; oc (T,X) X it be uniformly overtaking optimal for problem ([ jq|) - (|ifo[) . 

Now, for an unbounded increasing sequence of times (r n ) n ^ G T N , there exist: 
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1) its subsequence (t n )neN £ T ; 

2) the sequence of initial conditions (Cn)neN £ X N that converges to Ox; 

3) the sequence (X n ) n £N £ [0, 1] N that converges to some X° G [0, 1]; 

such that if ip° G C(T, X) is defined for every t G T by the rule 

n, s /"*" dq{t,x c (t),u°(t)) , s i 

t/ien, t/ie /irmt would be uniform on every compact, and (x°, it , A , V' )rieN € 3 uiould 
satisfy all relations of the Maximum Principle t\10a$ - <\10d\ . Moreover, 

n, s r, f tn 9g(t,x r (t),u°(t)) , s ,. , 

/ T = lim A" ^ ' C " V ; ' — y -^A c (t )dtA7 1 {T) VT G T. (37) 

n— ¥oo Jj, OX ^" 

Proof. Indeed, consider the control system (p, q) = (b,c) = a from ()30[) . It features 
the set of controls 11, however, as a system of form ([3]), it defines the control 
system of form ^ that is controlled by the elements of it. For such a system, fix 
the weight w° from the formulation of Proposition [3J 

By Remark[31 for every (r n ) ne N G T N , there exist its subsequence (t n )neN £ 
T and the sequence 

{x n ,r) n , A",V")„eN £ 2) of solutions of system GHaHI50, 
converging to the certain solution (x , ur,X , if} ) £ 3 of all relations of the Maxi- 
mum Principle. 

Now, for every n G N, we can find B n G C(T,L) and p n G C(T,X) such that 

a" = (p„, J B„,x",A")G2l[ 77 ™], B„(0) = 1 L , p„(0) = V"(0). (38) 

On the other side, differentiating ip n B n (as in (129(1 ), we check that 
(4> n B n ,B n ,x n ,X n ) G 2l[»7 n ]. Comparing the initial conditions, we see that p n = 

^ n B n . 

For each n G N, for each f 6 T, there exists the solution a n ,t G C(T,E) of 
(|31[) for the initial conditions a Mi t(0) = x(t,a n (t)). Note that the last components 
of a n ,t and a™ are independent of t; thus, they correspond with A™. Now we can 
correctly define the components of the map t n> x(i, a n (t)) by the rule 

x{t,a n {t)) = (u n {t),^ n {t),C l {t),\ n ) Vt G T, n G N. 

Substituting these initial conditions into (|32|) . by virtue of equalities (|38ll and 
a n (t) = a n , t {t) for all n G N, t G T, we obtain 

(ip n (t)Bn{t),B n {t),x n (t)) =a n ,t(t) = (v n (t) - A"J ? «( t )/i„(i),A e »(t)/i n {f),a; ? n( t )). 

Specifically, 

^"(*)A e(t) (i) = Ip™ (t) B n (t) [In 1 (t) = Mt)Vn\t) - A"J e(t) (f). 

Remark U] provides ip n (t n ) = 0; substituting i = i„, we obtain 

= V n (tn)A e n (t „)(i n ) = Unitn)^ 1 ^) - X n (t n )I^ {tn) (t n ). 
Substracting one from another yields 

i> n (t)A iHt) (t) = Vn(t)Hn\t) - X n I iHt) (t) - Vnitn^itn) + X n I^^tn) . (39) 
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By RemarkU we have a™(0) = (ip n (0), 1 L , X , A") -> (V>°(0), 1 L , X , A ), and 
l|£iu ( 7 ?™)llc -> as n -> oo; moreover, Proposition [3] yields for all t £ T 

max||a t ,„(0)-a n (0)|| £ =max||x(t,a n (f))-x(0,a n (0))|| B < ||£ w o (*?")! |c ^ 0. 

Hence uniformly on the whole T as n — > 00, it holds that 

a*,„(0)= (i^(t),A*n(t)^ n (*),A n ) ->(V> (0),1l,0x,A ) VtGT. (40) 

Whence the theorem of continuous dependence on initial conditions yields the 
uniformity of the limits 

Urn Vn^Hn 1 ^) = V>°(0), lim A n I*» M (t) = X°I (t) Vt G K, r G T. 

as n — > 00 for each compact A' g (comp)(T). Putting here r = i, r = i n , let us 
consider the limit of both sides of (|39p asn->oo; thus, 

■^ {t)A {t) = lim ^ o (0)-A o / o (t)-V O (0) + A"/ c „ (tri) (t n )) = 

n— too \ v 7 / 

lim (-^, w (() + A"/ { , (y ((„))= lim A n (l € « (tn) (t„)-I c » (tw) (t)). 

Multiplying on the right by Aq 1 (t) = A* (t) and Ao(t)A^nu)(t) we obtain our 

proposition for C, n = i n {t n ). All necessary convergences are provided by uniformity 
of limits in (gDJ). □ 
We say an optimal pair (x°,u°) G Q oc (T,X) x il for problem (fTa|) - (|lb[) is 
abnormal if every solution (x°, u°, A , ip°) G 3 of all relations of the Maximum 
Principle (fT0a|) -(fT0d |l satisfies A = 0. 

Remark 7 Assume conditions (u), (fg), (<9). Let the pair (x°,u°) G C; oc (T,X) x il 
be uniformly overtaking optimal pair for problem (|lap - (llb[) and let this pair be 
abnormal. Then, 

lim sup 1 1-^(7") I Is = lim sup 

Indeed, if it is wrong, then, the right-hand side of (|37p equals zero for T = 0, i.e., 
ip°(0) = X , which contradicts the relation (|10d[) for A = 0. 

5.3 Monotonous case 

Consider the case of when both the right-hand side of the dynamics equation and 
the objective function are monotonous. This case frequently arises in economical 
applications while monotonicity simplifies its examination. It seems that the first 
to note the peculiarities of this case and to investigate it were Aseev, Kryazhimskii, 
and Taras'ev in their paper [B], These were followed by papers [35] , [1] , [2] , and the 
most general case was considered in [3] . 

In Euclidean space E' , let us define binary relations y~ and y by the rules 

(a y- p) (a - e T dmE ), {ay 0) «*■ (a - e Rf^ E ) Va, f3 G E 1 . 

This allows us to use the symbols y- and >- to compare vectors and matrices, and 
vector and matrix functions. For the latter two, y- and y allow us to discuss their 
monotonicity. 



T %(t,^(t), M u (t)) 
dx 



A^(t) alt 
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Proposition 8 Assume conditions (u),(fg),(9) hold. Let the pair (x°,u°) G 
C/ oc (T,X) x il be uniformly overtaking optimal for problem (|_?a[) - (| _?&[) . Assume for 
all x G X and for a. a. t G T there exists a number d(t,x) G R such that the following 
relation holds: 

dg(t,x,u°(t)) df(t,x,u°(t)) 

dx dx ^"(.i,a;JlL- 

Then, there exists a solution (x° ,u° , X° ,ip°) G 3 of all relations of the Maximum 
Principle $TUa} - j 1 Oa} satisfying OTp, and ifj° y. X - 
If at the same time the pair (x u , u ) is normal, then 

\° limsup I ( (t) y- V°(0) ^ A lim J (i) !>= X (41) 

i-s-oo.C-s-0 '-* 00 
Zio/rf, and all limits in well-defined and finite. 

Corollary 2 Assume conditions (u),(fg),(9) hold. Let the pair (x°,it°) G 
C/ oc (T,X) xit be uniformly overtaking optimal for problem (I la$ - \lb\ . and let this pair 
be normal. Assume for all x G X and for a. a. t G T there exists a number d(t, x) G R 
such that the following relation holds: 

dg(t,x,u°(t)) df(t,x,u°(t)) 

dx 9-r J-l,- 

Then, there exists a solution (x°, n°, A , G 3 of all relations of the Maximum 
Principle §TUa} ~ §TUa\ satisfying and ip° y X - 

Proof. Below, in the proof of Proposition^ we understand the symbol t> as and 
in the proof of Corollary [2] we understand it as y. 

Fix arbitrary £ G X,T G R>o,t £ <T,oo); let us show that J 4 ? (r)A^ 1 (T) t> L . 

Denote by F f (t) the matrix g/fe^gil^W) f or a n t e [r, T ]. The diagonal of the 

map is dominated by a function M = M^ T ^ G £j oc (T,T); then, by condition, 
+ m(t)lL|(T !r ] O Ol- Now, let us consider a solution P(t) of the equation 

P = (F^t) + M(t)l L )P,P(T) = l Ll t > T; 

for it, it holds that P(t) O Ol for all t G (T, r]. Since A^ and 1l commute, the 
solution P is the product of two solutions of the equations Q = F^(t)Q, Q(T) = ±l, 
and R = M (t)l L P, R(T) = 1 L . Thus, 

P( r ) = Q(r)P(r) = Q(r)e^ = A ^ T ) A ^{r) e f T M(t)dt ^ 

and P(t) O Ol implies yl 5 (r)A^ 1 (T) O Ol for all r G (T, oo). Now, by monotonicity 
of matrix product, we obtain 

dlf(t) dq(t,xAt),u°{t)) i, . 

-J±lAf(T)= ^' ^J' V " A^t)Af(T) > Ox Vt€<T,oo) (42) 

for all £ G X, T G T; specifically, for T = we have > Ox, hence the functions 

1^, I^AT 1 ^) are monotonically increasing for all £ G X, T G T. 
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By Proposition [7] there exists the solution (a; , u , A ) of relations of the 
Maximum Principle satisfying of formula (|37[) for certain sequences A" and £„. 
However, the expression into the limit of (|37p lies in L^ by (|42p . Passing to the 
limit as n — >■ oo, we obtain ip° )p Ox- 
Suppose the pair (x ,u ) is normal; then A > 0. Since the function 1^ is 
monotonically increasing and, by Remark uniformly bounded in the certain 
neighborhood Ox, the Lebesgue theorem yields the existence of the finite limits 
in (f4"Tj) . Hence, 

A limsup Ut) )? A lim I((t n ) = V°(0). 

t->-oo,f->-0 n ^°° 

On the other side, monotonicity of I^A^ (T) yields 

^°(T)^ lim (l u (t n )-I Cn (T))A^(T)9 lim (i^Jt) - I in (T))A^(T) 



= (/o(t)-/o(T))i 1 (T) > X VreT,t£(T,oo), 

i.e. ip° t> Ox- Moreover, substituting T = and passing to the limit as t — > oo, we 
obtain the lower estimate from (|4ip . □ 

Note that in [6l Theorem 1], gl Theorem 10.1] the estimate t/j ^ X >- X ) 
is made for autonomous systems under less general assumptions; in [11 Theorem 
10.1], the lower estimate from (|4ip was made too (see [H (10.17)]). However, in 
these papers, the condition A > was not assumed but proved; namely, with the 
aid of the normal-form stationarity condition, the boundedness of integrals of (I37|) 
was proved, which guaranteed the control was normal. 

Let us also note that formula (|35[) was also proved for biaffine control system 
for monotonic ff (01 Theorem 1],[1 Theorem 11.1]). It seems, this result is not a 
direct consequence of Theorem [2l which was proved in that paper. 



6 Addendum 

In the paper, the left endpoint is considered to be fixed. It seems this condition 
may be easily discarded, since to do it, it is sufficient to equip the finite horizon 
optimization problems from the proof of Theorem [1] with the same condition for 
the left endpoint and to provide the boundedness of a;(0). 



6.1 Case of a-compact-valued map U 

The condition (u) implies that at every time ( 6 T, the controls are chosen from 
the compact U(t). Let us weaken this assumption to the following: 

Condition (u 5 ) : U is a a-compact-valued map such that GrU € 13 (T x U). 

We shall still assume the conditions (a) , (fg) to hold, and we shall not change 
the definition of il. Then, we can assume there exists a nondecreasing sequence 
(t/*- r )) r6 N of integrally bounded (on each compact subset of T) compact-valued 
maps such that U = Ur^U^. Let us assume the uniformly overtaking optimal 
control u exists. Then, we may safely assume that Gru° C GrU {1 \ 
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Repeating the reasonings of Sect [2] for each r G N, we can construct sets 

il (r \il (r > and their images for the restriction: = 7r„(it (r) ),i& r) = 5F„(H (r) ). 
Let us introduce the set il of all maps 77 from T into the set of Radon probability 
measures over U such that for every n G N there exists r = r(rj,n) G N such that 
^n(rj) = ??l[o,n] £ M&P • The topology of this set is of no use to us, thus we assume 
it is indiscrete. Note that under our definition, 5(H) <£ it, but u° € <5(il (r) ) C 5(H) 
for all r G N 

Note that, for all 77 € il, the set 2l[?j] is still compact. Indeed, for each n G N, 
it holds that 2t[r?]| [0:Il] C S^^'^^lio.n] G (comp)(C([0,n], E)); all that remains 
is to use the definition of compact-open topology. For each r G N, denote by 
Z^ the pairs (ip,X) G C(T,X) such that (x° ,u° ,ip r , X r ) satisfies relations (fTOafl- 
(|10bl) . (|10d|) . and for a. a. t G T instead of ()10c|) . there holds the weaker relation 

sup H(x(t),t,p,X,^{t)) = H(x(t),t,u°(t),\,il}(tj). (43) 

Note that this set is compact (it follows from compactness of 21(77] ). By Theoremfll 
is not empty for all r G N. It is easily seen that Z^ r ' C Z^ r ' for any 
r',r" G N, (r' C r"). Then, there exists , X°) G rV eN Z (r) . Therefore, for it, (g3J) 
holds for all r G N; thus, (|10c[) holds too; hence, (x , it , t/>°, A ) satisfies all relations 
(|10ap - (|10d|) of the Maximum Principle. 

For each r G N, consider the sequences (x™, 77™, A™, )neNi (*n,r) n eN from Re- 
marklU Then, for the sequence (x„,rjn, A™, V , n)neNi by uniformity of estimate (|14l) . 
we have pointwise convergence of r\ n to So; moreover, for each k G N in the interval 
[0, fc] for this sequence, the convergences from Remark [3] hold (it is sufficient to 
consider there topologies with respect to C([0, k], X),il^ fc ^), specifically, it would 
hold that 

1) (x n , n n ,K,^) -»• (a; ,,! , A°,7/,°)„ eN G C Joc (T,X) x£x T x Q oc (T,X); 

2) |^(77")||c^O; 

3) Jt n (v) — Jt n ( u ) ~~ 0j an d i>n{tn) = for each n G N, where £„ = i„,n- 
The verbatim repetition of the proof of Proposition [2] yields 

Proposition 9 Assume conditions (u<j), (fg). For each uniformly overtaking optimal 
pair (x°,u°) G C; oc (T,X) x il satisfying (tp) for each unbounded increasing sequence 
( 7 n)neN £ T N there exists (x°, u° , A , t/>°) smc/i t/iaf f/ie relations of the Maximum 
Principle t\10a^ - <\10dfi . and the transversality condition §15b\i hold. 

Since our case is more general, starting with Sect.|4l the references to (u) ought 
to be replaced with (u,), and the results of Sect. [3] ought to be replaced with their 
respectful analogues. 



6.2 On uniformly sporadically catching up controls. 

Definition 3 We say that a control it G il is uniformly sporadically catching up 
optimal if for every e, T G Ft>o there exists t G [T, 00) such that J±(u ) > Jt(u) — e 
holds for all u G il. 
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Note that for each uniformly sporadically catching up optimal control, there exists 
an unbounded monotonically increasing sequence (t„)„ 6 n £ T N and a function 
lu° 6 fl such that 



We call such control a r-sporadically catching up optimal. 

Now, if we consider the sequence (T n ) n6 N everywhere defined and understand 
the optimality in the above sense, then all statements, starting with Theorem [TJ 
hold. In particular, we can rewrite Proposition [6] and Theorem [2] in the following 
way: 

Theorem 3 Assume conditions (u CT ), (fg), (d) hold. Let the pair {x ,w ) £ 
C/ oc (T,X) xil be t -sporadically catching up optimal for problem (| jqp - (|Jb|) . Let J, gX 
be the limit of I^(T n ) as n — > oo,£ — > Ox- 

Then, there exists the unique solution (x ,u° , \° ,ip°) of all relations of the Maxi- 
mum Principle t\10a\) - (| Wtty and the transversality condition §25b\ . Moreover, accurate 
to the positive factor, we can assume 
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